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Abstract 

Across the United States, many faculty members are developing new mixed re¬ 
search courses. However, before embarking on teaching these courses, it would be 
helpful for instructors to be aware of the challenges faced by instructors and stu¬ 
dents in mixed research courses. Thus, the purpose of this qualitative-dominant 
mixed research study was to document these challenges. Participants were 11 in¬ 
structors of mixed research courses from institutions around the United States 
who were selected purposively via critical case sampling such that they represent¬ 
ed a diverse set of instructors. The following four themes emerged that represent¬ 
ed dimensions of challenges: Time, Diversity, Format/Life Situations, and Pre¬ 
conceived Bias. Three of these themes yielded 10 subthemes. Moreover, a corre¬ 
spondence analysis of the four themes revealed that they each fell somewhere on 
a continuum that lay from internal influences that characterized an interaction be¬ 
tween instructors and students (i.e., Diversity, Format/Life Situations) to external 
influences that characterized an interaction between instructors and curricula per¬ 
taining to the rapidly evolving field of mixed research—yielding the meta-themes 
of Internal Influence and External Influence. Implications for instructors and stu¬ 
dents undertaking mixed research courses are discussed. 

Keywords: Mixed research; mixed methods research; mixed research courses; mixed re¬ 
search pedagogy; pedagogical challenges. 


The overwhelming majority of doctoral students representing the social and behavioral 
sciences are required or expected to complete at least one research methodology course 
as part of their degree programs (Capraro & Thompson, 2008; Leech & Goodwin, 2008), 
and although these research methodology courses tend to represent either quantitative 
research courses or qualitative research courses, in recent years, an increasing number of 
students are being exposed to mixed research courses—wherein quantitative and qualita¬ 
tive research approaches are taught within the same course. For example, Leech and 
Goodwin (2008), who surveyed 100 schools of education across the United States, docu- 
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mented that 22% of programs required students take a mixed research course and 20% of 
programs offered mixed research as an elective. And, bearing in mind that the data for 
this study were collected more than 4 years ago, it is very likely that the number of mixed 
research courses being taught worldwide has increased significantly, especially consider¬ 
ing the increased visibility of this third methodological movement through such venues as 
journals devoted to mixed research (i.e., Journal of Mixed Methods Research ; Interna¬ 
tional Journal of Multiple Research Approaches), conferences devoted to mixed research 
(for example see http://www.healthcareconferences.leeds.ac.uk/conferences/) , handbooks 
(i.e., Tashakkori & Teddlie, 2010), books (e.g., Bergman, 2008; Collins, Onwuegbuzie, 
& Jiao, 2010; Creswell & Plano Clark, 2010; Greene, 2007; Hesse-Biber, 2010; Johnson 
& Christensen, 2010; Morse & Niehaus, 2009), and mixed research articles published in 
high-impact journals (e.g., Johnson & Onwuegbuzie, 2004). 

As surmised by Creswell, Tashakkori, Jensen, and Shapley (2003), “very few courses are 
currently available specifically on mixed methods research” (p. 620). Indeed, most of the 
existing mixed research courses have emerged in the last 6 years. As a result, the vast ma¬ 
jority of instructors of mixed research courses never took a mixed research course them¬ 
selves—unlike the vast majority of instructors of quantitative-based (e.g., statistics cours¬ 
es) and qualitative-based research courses. Thus, instructors of mixed research courses 
represent what Creswell et al. (2003) referred to as a 'first generation of faculty” (p. 
620). Unfortunately, as noted by Earley (2007), instructors of mixed research courses 
“find ourselves in the same situation: we were not officially trained in the mixed-methods 
research process and have to create these courses without the benefit of prior coursework 
to guide us” (p. 146). 

As declared by Onwuegbuzie, Frels, Leech, and Collins (2011), the greatest challenge 
faced by the first generation of mixed research course instructors concerns the lack of 
works published in the area of mixed research pedagogy. Interestingly, Frels, Onwueg¬ 
buzie, Leech, and Collins (2012) found only 19 published works that were devoted pre¬ 
dominantly or exclusively to the topic of teaching mixed research-based courses, which 
represented less than 2% of the mixed research works published between 2000 and 2009 
that were identified by Ivankova and Kawamura (2010). Of these 19 works, only two of 
them (Creswell et al., 2003; Ivankova, 2010) provided substantive information about the 
challenges faced by instructors of mixed research courses. 

Creswell et al. (2003) discussed pedagogical challenges that they labeled as: textbooks, 
mentoring/support, inconsistent terminology, personal bias, anxieties/misconceptions, 
and design and data analysis. With regard to textbooks, Creswell et al. discussed the pos¬ 
sibility that finding an appropriate mixed research textbook might be challenging. How¬ 
ever, with more than a dozen mixed research textbooks currently in existence, it is much 
less likely that this challenge still prevails. Mentoring/support refers to the lack of “well- 
established support systems of colleagues from which to draw” (p. 630). However, with 
the emergence of online mixed research discussion groups (e.g., 
http://www.linkedin.com/groups/Mixed-Methods-Research-3794214 ; 
http://www.methodspace.com/group/mixedmethodsresearchers) and special interest 
groups (e.g., American Educational Research Association Mixed Methods Special Inter- 
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est Group; http://www.aera.net/Default.aspx7menu id=524&id= 10668) , several outlets 
exist for instructors to receive support. 

Further, the use of inconsistent terminology in mixed research represents another chal¬ 
lenge to mixed research course instructors that was discussed by Creswell et al. (2003). 
Personal bias, another challenge, involves the learning of multiple philosophical assump¬ 
tion and stances. Anxieties/misconceptions refer to all of the challenges and frustrations 
associated with quantitative and qualitative methods that students bring to mixed research 
courses. Finally, with respect to design and data analysis, Creswell et al. (2003) stated 
that “perhaps the biggest challenge instructors face when teaching mixed methods re¬ 
search is teaching students which design to use and how to analyze the data once they 
have been collected” (p. 632). 

Similar to Creswell et al. (2003), Ivankova (2010) provided some evidence-based peda¬ 
gogical challenges faced by one instructor of an online mixed research course. In particu¬ 
lar, she discussed the challenge stemming from the teacher having to “rely on her own 
experience and student feedback in designing and teaching this online mixed methods 
course due to the lack of practical advice and related discussion in the literature” (p. 59). 
Another challenge represented finding the appropriate balance between reading and writ¬ 
ing assignments. A third challenge stemmed from the goal of including a large amount of 
material into one mixed research course. Other pedagogical challenges discussed by 
Ivankova stemmed from the lack of resources that would serve as foundational readings 
for the onhne mixed research course, lack of a comprehensive mixed research textbook, 
having to rely on multiple reading sources, the diverse perspectives on mixed research, 
the varied terminology, and scarcity of published empirical mixed research studies in dif¬ 
ferent disciplines that can be used as examples to guide students in their research pro¬ 
posals. 

Although both the Creswell et al. (2003) and Ivankova (2010) works contain excellent 
information about pedagogical challenges, to date, no evidence-based challenges have 
been documented for face-to-face mixed research courses across various settings. To this 
end, the purpose of the present mixed research investigation was to examine the chal¬ 
lenges faced by instructors who taught a variety of mixed research course formats, and to 
examine the extent to which these challenges are related to instructors’ mixed research- 
based philosophical assumptions and stances. 

Research Questions 

Qualitative research questions. The following qualitative research questions were ad¬ 
dressed in this study: 

1. What are the pedagogical challenges in mixed research courses faced by selected 
U.S.-based leading mixed methodologists? 

2. What are similarities and pedagogical differences in mixed research courses in 
challenges faced by selected U.S.-based leading mixed methodologists? 
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Mixed research question. The following mixed research question was addressed in this 
study: 


What are the pedagogical challenges in mixed research courses faced by selected 
U.S.-based leading mixed methodologists as a function of conceptual stance? 

Theoretical Framework 

According to Teddlie and Tasha kk ori (2010), there are six contemporary conceptual 
stances associated with mixed research: a-paradigmatic stance, substantive theory stance, 
complementary strengths stance, multiple paradigms stance, dialectic stance, and alterna¬ 
tive paradigm stance (formerly called single paradigm stance). Each of these stances is 
summarized in Table 1. As stated by Teddlie and Tashakkori (2010), these six conceptual 
stances have “been used (explicitly or implicitly) by groups of scholars who are practic¬ 
ing MMR [mixed methods research]” (p. 14). Interestingly, Onwuegbuzie et al. (2011) 
documented a relationship between the conceptual stance of mixed research course in¬ 
structors and their pedagogical approaches. Thus, we hypothesized in this study that the 
instructor’s conceptual stance (i.e., mixed research lens) would play an important role in 
the types of pedagogical challenges. As such, we believed that conceptual stance of 
mixed researchers provided an appropriate theoretical framework for this inquiry. 

Method 


Participants and Setting 

Data collection for this mixed research study, part of a larger study investigating the ex¬ 
periences of instructors and students in mixed research courses, took place either face-to- 
face at various national and international conferences or remotely (e.g., via phone or In¬ 
ternet). The participants were 11 leading mixed methodologists, five men and six women, 
from various institutions in the United States who were instructors of mixed research 
courses. They were selected via a criterion sampling scheme (Bernard, 2000; Miles & 
Huberman, 1994; Onwuegbuzie & Leech, 2007). As noted by Guest, Bunce, and John¬ 
son (2006), 12 interviews are sufficient to understand experiences and perceptions among 
a common group and as posited by Johnson and Christensen (2010), “when greater re¬ 
sources are available, collective case studies of around 10 cases are common” (p. 397). 
Therefore, we deemed the sample size of 11 instructors adequate for obtaining data satu¬ 
ration. The 11 participants taught a variety of mixed research course formats— 
specifically, (a) in a site-based 16-week semester format, (b) in a site-based condensed 3- 
weekend format, or (c) in an on-line 16-week semester format. Using the Carnegie Clas¬ 
sification (The Carnegie Foundation for the Advancement of Teaching, n.d.), the instruc¬ 
tors’ affiliations were as follows: institutions with very high research, institutions with 
high research, institutions with doctoral-level research, or institutions wherein research is 
not classified. 
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Table 1. Tashakkori and Teddlie’s (2010) Six Conceptual Stances Associated with 
Mixed Research. 


Conceptual Stance 

Description 

A-paradigmatic 

Paradigms or conceptual stances are not important to 
real-world practice 

Substantive theory 

Theoretical orientations (e.g., critical race theory) are 
more pertinent to the underlying research study than 
are philosophical paradigms 

Complementary strengths 

Mixed research is possible but the different approach¬ 
es must be kept as separate as possible in order for the 
strength of each paradigm to come to the fore 

Multiple paradigms 

A single paradigm is not appropriate for all mixed 
research designs; rather, different paradigms are rele¬ 
vant for different mixed research designs 

Dialectic 

Use of multiple paradigms in a single mixed research 
study yields greater understanding of the underlying 
phenomenon 

Alternative paradigm 

Single paradigm (e.g., pragmatism-of-the-middle; 
transformative emancipator) is used to support the use 
of mixed research 


Per Teddlie and Tashakkori’s (2010) framework, the mixed research conceptual stances 
of the 11 participants were as follows: five participants were classified as endorsing a dia¬ 
lectic stance, four participants were classified as supporting the alternative paradigm 
stance, one participant was classified as promoting the multiple paradigms stance, and 
one participant was classified as advancing the complementary strengths stance. Conse¬ 
quently, four of Teddlie and Tasha kk ori’s (2010) six conceptual stances were represented 
by the 11 participants. Table 2 presents each participant’s associated conceptual stance. 

Instruments and Procedures 

After Institutional Review Board approval was obtained, two researchers conducted the 
series of interviews as the means to obtain data. These interviews were audio-taped using 
two separate hand-held digital recorders to insure clarity of recordings. The interviews 
were semi-structured in nature, consisted of open-ended questions, and included probing 
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Table 2. The 11 Participants and Associated Conceptual Stance. 


Conceptual Stance 

Number of Partici¬ 
pants Identified in 
Stance 

identifying Name for 
Each Participant 

Alternative paradigm 

4 

Participant API 

Participant AP2 
Participant AP3 
Participant AP4 

Dialectic 

5 

Participant D1 

Participant D2 

Participant D3 

Participant D4 

Participant D5 

Complementary strengths 

1 

Participant CS1 

Multiple paradigms 

1 

Participant MP1 

A-paradigmatic 

0 


Substantive theory 

0 



questions to obtain rich data (Geertz, 1973). Samples of interview questions are as fol¬ 
lows: 

1. What are the particular issues that students face in learning about mixed methods? 

2. What are the issues for teachers in designing and delivering courses that aim to 
develop researchers’ abilities to carry out mixed methods? 

After interviews were transcribed, the participants were asked to complete member 
checking (Lincoln & Guba, 1985) in order to maximize descriptive validity (Maxwell, 
1992). Further, the two researchers who were involved in interviewing the participants 
underwent debriefing interviews themselves (Frels, Onwuegbuzie, & Frels, 2010) be¬ 
cause they served as primary research instruments (Poggenpoel & Myburgh, 2003). De¬ 
briefing interviews, as conceptualized by Onwuegbuzie, Leech, and Collins (2008), are 
designed to promote reflexivity; to identify biases in interpretation of data; and to obtain 
rich insights as to ways that the study impacted participants, stakeholders (i.e., instructors 
and students of mixed research courses), and the researchers themselves. 
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Mixed Research Design 

In the current study, we utilized a qualitative-dominant mixed research design (Johnson, 
Onwuegbuzie, & Turner, 2007), wherein we adopted a qualitative, constructivist- 
poststructuralist-critical stance with respect to the research process, while, at the same 
time, deeming the inclusion of quantitative analyses to yield value-added inferences. 

The embedded design utilized in this inquiry was a multiple case study (Yin, 2009)—also 
known as a collective case study (Stake, 2005). According to Stake (2005), a multiple 
case study represents an instrumental case study with multiple cases/participants who are 
instrumental to the study because the information they yield provides insight into the un¬ 
derlying phenomenon—in this case, the phenomenon of challenges in teaching mixed 
research courses. Specifically, an embedded design was utilized, wherein the results of 
each case were understood through cross-case analyses (Yin, 2009). The cases were 
bound together by the fact that participants were leading mixed methodologists who 
taught mixed research courses. 

Data Analysis 

In this study, we adopted a dialectical pluralist stance with respect to the research pro¬ 
cess, wherein we incorporated multiple epistemological perspectives within the same in¬ 
quiry (Johnson, 2011). With respect to our data analysis phase, we utilized constant com¬ 
parison analysis (Glaser & Strauss, 1967), classical content analysis (Berelson, 1952), 
word count (Leech & Onwuegbuzie, 2007, 2008), and keywords-in-context (KWIC; 
Fielding & Lee, 1998) via the software QDA Miner Version 4.0 (Provalis Research, 
2011). The sources for naming codes and locus of typology (i.e., theme) development 
were investigative (i.e., stemming from the intellectual constructions of the researchers 
[Constas, 1992]) in an iterative process involving a posteriori coding. Also, the verifica¬ 
tion component was technical (e.g., use of intercoder agreement). Specifically, two of the 
researchers independently coded 20% of the interview data and after establishing 100% 
interrater reliability, using Cohen’s Kappa measure (Siegel & Castellan, 1988), one re¬ 
searcher coded the remaining interview data. After naming codes, we conducted a Jac- 
card’s coefficient to determine the co-occurrence of codes across and within each of the 
11 cases. Also, a case-oriented analysis was utilized, with the focus on the high frequency 
themes and subthemes. 

According to Sandelowski (2001), a narrative description can be enhanced via a word 
count. The word count list was condensed by eliminating words not specific to mixed re¬ 
search and challenges of teaching. In addition, KWIC was used to supplement the word 
count analysis. According to Leech and Onwuegbuzie (2007, 2008), the purpose of 
KWIC is to indicate how words are used in context with other words. As noted by Field¬ 
ing and Lee (1998), KWIC represents an analysis of the culture of the use of the word. 
Moreover, “the assumption underlying KWIC is that people use words differently and, 
thus, by examining how words are used in context of their speech, the meaning of the 
word will be understood” (Leech & Onwuegbuzie, 2008, p. 594). 
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These analytical procedures relied heavily on the discretion of the researchers, as a partic¬ 
ipative element of the process (Constas, 1992) and served as a means for maintaining an¬ 
alytic integrity (Miles & Huberman, 1994). First, the coding procedure considered three 
components outlined by Constas (1992): (a) origination, which is the programmatic lan¬ 
guage, the investigation, review of literature, or interpretations of the data; (b) verifica¬ 
tion, which is the evidence that the categories have a logical connection and can be veri¬ 
fiable in existing research; and (c) nomination, which is the process of naming the cate¬ 
gories and resulting in logical connections. Categories that emerged were investigative 
and codes were named based on the analysis, conducted a posteriori (Constas, 1992), us¬ 
ing the language of the participants (i.e., folk terms; Spradley, 1979). 

As part of the cross-case analysis, the subthemes extracted from the interviews were sub¬ 
jected to a correspondence analysis. Broadly speaking, a correspondence analysis is an 
exploratory multivariate technique involving factoring categorical (i.e., nominal level) 
variables and mapping them in a property space that displays their associations in two or 
more dimensions (Michailidis, 2007). 


Results 

The constant comparison analysis identified 10 emergent sub themes that clustered into 
the following four themes: (a) Time, (b) Diversity, (c) Preconceived Bias, and (d) For¬ 
mat/Life Situations. It should be noted that the third theme (i.e., Preconceived Bias) stood 
alone yet co-occurred indirectly to the other three themes. Therefore, we present this 
theme as a stand-alone theme. Figure 1 presents the four themes as they relate to each 
other. Table 3 presents the themes and subthemes from the constant comparison analysis, 
descriptions, and significant statements. 


Application of Knowledge 
Time Restraints 
Changing Field 

Different Fields of Study 
Diverse Levels of Students 

Language and Vocabulary 

Attention to Feedback 
Personal Life Situations 
Content for Online 

Preconceived Bias 




“i 


Agglomeration Order: Jaccard’s coefficient (Occurrence) 

Figure 1. Four themes that emerged as dimensions of challenges pertaining to teach¬ 
ing mixed research courses. 

The Journal of Effective Teaching, Vol. 12, No. 2, 2012, 23-44 

© 

2012 All rights reserved 



























Challenges to Teaching Mixed Research Courses 


31 


Table 3: Meta-Themes, Themes, and Definitions of Challenges of Teaching Mixed 
Research. 


Theme 

Subtheme 

Definition of Challenge 

Time 

Application of Knowledge 

Teaching students methodology specific 
to mixing quantitative and qualitative re¬ 
search (e.g., sampling, data analysis, inte¬ 
gration of quantitative and qualitative ap¬ 
proaches) 


Time Restraints 

Covering methodological concepts of 
quantitative and qualitative as well as 
mixed approaches in one course 


Changing Field 

Using current literature specific to strong 
mixed method design in a newly evolving 
research tradition 

Diversity 

Diverse Levels of Students 

Addressing in one course students who 
are stronger in one research approach 
(e.g., qualitative) versus the other ap¬ 
proach (e.g., quantitative) 


Language and Vocabulary 

Teaching research language pertaining to 
research in general and terms specific to 
mixed methods (e.g., convergent parallel 
design, pragmatism) 


Different Fields of Study 

Developing a course that meets the needs 
of students from various departments 
within the university 

Preconceived 

Bias 


Helping students overcome predetermined 
misperceptions pertaining to one approach 
(e.g., qualitative) versus the other ap¬ 
proach (e.g., quantitative) 

Format/Life 

Situations 

Attending to Feedback 

Changing each course presentation from 
semester to semester to increase students’ 
understanding and application of mixing 
approaches 


Personal Life Situations 

Addressing the typical students issues that 
arise but that are more intense due to the 
complexity of course content 


Online 

Adapting multifaceted concepts so that 
self-directed online learning is successful 
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As can be seen in the occurrence of subthemes in Figure 1, when participants mentioned 
the application of knowledge, often they discussed closely the time restraints relating to 
the amount of material to present and to apply. These two subthemes also connected to 
the idea that the field of mixed research is changing quickly as it evolves, comprising the 
theme of Diversity. In the theme of Time, the subtheme of Diverse Levels of Students 
was linked to the sub theme Language and Vocabulary, as well as the sub theme of Differ¬ 
ent Fields of Study. In the theme Format/Life Situations, subthemes of Attending to 
Feedback, Personal Life Situations, and Online were clustered as they related to the hu¬ 
man considerations pertaining to the challenges of teaching mixed research. Finally, as 
shown in Figure 1, the theme of Preconceived Bias has a dotted line linking it to each of 
the other three themes. The following sections present the four themes representing the 
four dimensions of challenges, which are presented in order of dominance (i.e., preva¬ 
lence rates among the participants’ voices), starting with the most dominant theme and its 
respective subthemes. 

Diversity 

The theme of Diversity comprised the subthemes of (a) Diverse Levels of Students, (b) 
Language and Vocabulary, and (c) Different Fields of Study. Diverse Levels of Students 
emerged as the most dominant subtheme when considering frequency (/7 = 31) and occur¬ 
rence (i.e., 8 of 11 participants). One instructor stated, 

My philosophy is that I am going to modify materials and lectures as necessary 
for the group of students and I often modify my expectations of what the students 
need based on what they're coming in with. I've got some students that come into 
introductory research that have no research background, and I have others that are 
coming into the advanced mixed methods class that have quite a bit of research 
background, but often in one strand. 

In addition, instructors often discussed philosophical challenges that related to Diverse 
Levels of Students and mixing research paradigms. For example, one instructor ex¬ 
plained, 

Exposure to research, it's not uncommon, where I am anyway, for students to 
come into the class, maybe they've taken two research courses and so they're kind 
of ill prepared to take a mixed research course and hit the ground running. Anoth¬ 
er issue that they face-again at my institution and several others where I've taught 
before-is lack of exposure to philosophy of research. It's very rare for me to find a 
student who is really aware, who has an identity in terms of research philosophy; 
that they know, for example, if they are a constructivist, or a pragmatist, or what 
have you, prior to the course. 

The subtheme of Language and Vocabulary was the second overall dominant subtheme 
with six of the 11 participants discussing it and an overall frequency count of 11. In addi¬ 
tion, most often when instructors mentioned the challenge of teaching diverse levels of 
students, they closely related this challenge to the diverse levels of vocabulary. Consider- 
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ing the language that is specific to research, it is not surprising that mixed research utiliz¬ 
es language from both quantitative and qualitative research traditions; subsequently, the 
vocabulary in mixed research courses would be quite extensive. One instructor explained, 

They just have to learn not only the content but also they have to leam the lan¬ 
guage, the terminology, and actually, the concepts you know. Content is more 
methodological, how you do things differently in mixed methods research, quanti¬ 
tatively or qualitatively, but concepts you know new things that are unique to 
mixed methods research for example like how you integrate. 

Pertaining to the subtheme Different Fields of Study (n = 4) within the theme of diversi¬ 
ty, four participants noted how often they found themselves challenged when working 
with students from different colleges within their universities. One participant noted, 

I think the biggest thing is that students come with varied backgrounds. My stu¬ 
dents come from multiple schools within the university. So I'll have nursing stu¬ 
dents, I'll have school of education students, I'll have school of public affairs stu¬ 
dents and, health and behavioral science students. 


Time 

The theme of Time was the second most dominant theme and comprised three sub¬ 
themes: (a) Application of Knowledge, (b) Time Restraints, and (c) Changing Field. Of 
these subthemes, Application of Knowledge (n = 18; across 8 of 11 cases) was most 
dominant. One participant explained, 

[Students] also struggle with separate topics, you know, specifically integration or 
mixing, legitimation issues, and that depends on the individual level and it de¬ 
pends on the coursework these students have or had in their research experiences. 

In addition, it was apparent that the majority of teaching challenges pertained to meth¬ 
odological concepts of mixed research—as another participant suggested, 

And I think, one of the things that I found is that students tend to not always 
flounder in the same area. So even though I can usually expect students to really 
struggle with the data analysis portion or the data integration portion, some stu¬ 
dents really have a hard time entering qualitative data and making numbers out of 
it. You know that they struggle with that. 

Finally, a third instructor explained the specific content that is unique to mixed research 
and, therefore, is a challenge: 

Well, if you're doing QUAL-QUAN when doing the QUAN component, there is 
an inadequate QUAL sample-there's not enough people, and not a randomly se¬ 
lected sample, to do the QUAN component. And if it's QUAN-QUAL, you have 
too many people from the QUAN component for the QUAL strategy, and you 
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have to really decide how you're going to select them. In a sequential design, it's 
even more problematic, because you can't find the subjects-they've gone away. 

These sampling problems are only the beginning. There are problems in the analysis, in 
the way you incorporate the results... 

A second subtheme of Time, specifically Time Restraints (n = 10), was discussed by four 
of 11 participants. Due to its complexity, mixed research was explained by participants as 
requiring much time. One participant described this challenge as follows: 

So I think that's like a big problem from the beginning. In terms of a second issue 
[students] have a hard time understanding how to conceptualize mixed methods, 
particularly in terms of writing the research questions, mixed methods research 
questions. There seems to be some problems there. Just in terms of trying to come 
up with an overarching problem that you're interested in and then coming up with 
complementary sets of quantitative or qualitative questions. So that usually takes 
a lot of time. 

In addition, the same participant explained, 

I also think that there's just not enough time in a typical semester to get the stu¬ 
dents to plan any well-designed individual (solo conducted) mixed methods re¬ 
search project in that period of time. So, the teacher has to decide whether they're 
going to have the students do a proposal, whether they're going to have them work 
in groups to try to actually gather some qualitative and/or quantitative data and do 
an actual study. 

The subtheme Changing Field (n = 9) was discussed by six participants. These instruc¬ 
tors, as leading authors in the field, were cognizant that current literature was critical to 
understanding the quickly evolving field of mixed research. One participant contended, 

I think there's a lot that is emerging now in mixed methods. And it is a field that 
we need to, be ready—quick on our feet I guess—to learn about what's emerging. 
It's not a field where you can take the old trusty textbook and say 'okay I got it out 
here.' And so the question really is 'how do we keep fresh with what's develop¬ 
ing?' And of course there are avenues, there are avenues. The Journal of Mixed 
Methods Research is great. 

In addition, finding credible articles was important to one instructor as he/she explained, 
“Are you asking me what I think is needed to better understand strategies for teaching 
missed methods? First, it is hard to find good examples—published, yes.” 

Preconceived Bias 

The theme of Preconceived Bias (n = 6 across four of 11 cases) refers to the way instruc¬ 
tors explained the mindset of students and the challenges associated to the preference of 
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one research paradigm over another (i.e., quantitative, qualitative). As explained by one 
instructor, 

So I do think there are still people who think either— we don't need mixed meth¬ 
ods because it's so simple, you know you just do this and you just do that and 
what do you have to learn about. Or, because they really truly believe they are sit¬ 
uated within one particular approach and that's sufficient, you know, you should¬ 
n't force or expect people to be competent across boundaries...sometimes people 
are very entrenched in the beliefs that their method is the one that's right. And, 
that, they can't really be an expert in everything so it's ridiculous to try. 

Another instructor noted how students might have been influenced by preconceived bias 
of former instructors: 

Attitudes—they [students] unfortunately come to mixed research classes with 
negative attitudes either from other instructors, professors, or from their peers 
with a negative attitude, and have a lack of appreciation of how powerful research 
can be in their fields. 

A third instructor reflected further on the challenge of teaching mixed research and pre¬ 
conceived bias and remarked, 

I think where professors of mixed methods research are the most vulnerable 
would be from criticism by qualitative researchers who say “How do you expect 
students to really understand the qualitative side of it if they haven't spent a sig¬ 
nificant amount of time out in the field actually experiencing, what it is that 
they're studying?” And I think that's a really good criticism. And so what I hope 
students would get is some sort of feeling of what that's about—that is what the 
qualitative end of mixed methods is about. 

As noted previously, the theme Preconceived Bias was not clustered with one specific 
theme, yet related to all three themes in some way (See Figure 1). One instructor ad¬ 
dressed the challenge of bias in the following manner: 

Definitely a lot is needed in teaching strategies because it's a new area. How do 
you teach mixed research? One of the problems we have in courses, in general 
mixed research courses, is that the instructor himself or herself may not be strong 
in both approaches, he or she may not be adequately strong in both quantitative 
and qualitative techniques so then there's a struggle; you know, what do you do? 
Well, obviously for me, one solution is to have people team-teach the course. So, 
you have two or more people team-teaching the course; so, maybe you have one 
or more that is quantitative oriented, other qualitative oriented and together stu¬ 
dents will get the best of both worlds. 
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Format/Life Situations 

In Format/Life Situations, subthemes of (a) Attending to Feedback, (b) Personal Life Sit¬ 
uations, and (b) Online referred to the personal challenges of students with respect to 
learning mixed research and challenges. The subtheme Attending to Feedback (n = 4; 
across four of 11 cases) referred to the challenge of providing feedback to various levels 
of students and various levels of groups of students as a whole. One instructor explained 
that because he/she used current readings due to the quickly changing field, preparation 
time can be extensive and attending to individual student feedback can be challenging: 

Every semester has different sets of readings. And I've actually found that one of 
the challenges that I faced the last semester was too orientated towards philoso¬ 
phy. I need to back out of that. Those are decisions that I make on a continual ba¬ 
sis. A lot of the modifications end up happening through a combination of asking 
students for feedback, and looking at the work they are doing. 

For another instructor, one challenge involved how best to amend the many concepts of 
mixed research after reflecting on one course cycle before moving to the next. He/she 
remarked, 

There's so much to cover in one course. You know it's sort of build from the 
knowledge that we have in QUAN and QUAL but there's still so much issues with 
mixed methods today. Students cannot get everything in one course that is needed 
for developing the proposal is too much. That is one issue that I always have— is 
to think at the beginning when each course began every year. I think, I sit down 
and look at my syllabus and think what would be the best logical way to present 
the topics? What should I change? And I change it every time, I change some¬ 
thing. 

Another subtheme, Personal Life Situations (n = 3; three of 11 cases) related the idea that 
due to the extensive nature of mixed research, coursework might take more of a personal 
toll on students. As explained by one instructor, 

Generally an issue I have is my high expectations and the rapid pace of the cours¬ 
es. It is a challenge for those that are working full time. It is very difficult. In the 
introductory class, it's [working students] generally is over 50%. At the doctorate 
level, I would say it's probably, 30 to 40% or something like that but I can't be 
sure. 

Closely related in Theme 3 to Personal Life Situations is the subtheme of Online (n = 2 in 
two cases overall). One participant who taught mixed research solely in an online format 
emphasized that time constraints due to personal lives are magnified and stated, “I also 
use a Tegrity program because some cases there are students who don't want to interact 
because of time constraints or because they can't all come together at that one moment.” 
He/she also noted with respect to understanding new concepts and online content that, 
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When considering the framework relative to use of terminology, and relative to 
recognition of different philosophical stances, influences, how one approaches 
mixing. I teach online so that I think that there's an added level of concern on the 
part of the students because they can't necessarily see me [in a face-to-face con¬ 
text]. 

Interestingly, Theme 3: Format/Life Situations overall appeared to be challenges that 
many traditional research courses might include. In short, research methodology courses 
can be challenging to students, and these challenges increase in mixed research courses 
due to the fact that mixed research coursework includes multiple aspects of both quantita¬ 
tive and qualitative traditions as well as the new content and terminology specific to mix¬ 
ing. 

Relationship of Themes 

Figure 2 presents the subthemes pertaining to challenges of teaching mixed research in 
summation, regardless of cluster, and overall frequency. 


Diverse Levels of Students 
Application of Knowledge 
Time Restraints 
Language and Vocabulary 
Changing Field 
Preconceived Bias 
Different Fields of Study 
Attending to Feedback 
Personal Life Situations 
Content for Online 



0 2 4 6 8 10 12 14 16 18 20 22 


Figure 2. Overall frequencies of themes pertaining to challenges of teaching mixed 
research courses. 
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Diversity theme and Format/Life 
Situations theme yielding an Internal 
Influence Meta-Theme 


Time theme and Preconceived Bias 
theme yielding an External Influ¬ 
ence Meta-Theme 


Figure 3. Correspondence analysis of 11 participants and relationship with each 
other and with respective themes. Also, the quadrants are numbered 1-4. 


Figure 3 displays the correspondence plot that emerged when the 10 sub themes were 
mapped onto a two-dimensional representation. As can be seen from this figure, the Pre¬ 
conceived Bias theme resided by itself on the upper right quadrant (i.e., Quadrant 2), in¬ 
dicating the uniqueness of this theme. Further, all the subthemes pertaining to the Diver¬ 
sity theme and all the subthemes pertaining to the Format/Life Situations theme were po¬ 
sitioned on the left side of the continuum of the horizontal axis (i.e., Quadrant 1 and 
Quadrant 3), whereas two of the three subthemes pertaining to the Time theme (i.e., 
Changing Field and Time Restraints) were positioned on the right side of the continuum 
of the horizontal axis (i.e., Quadrant 2 and Quadrant 4). Thus, the left side of the continu¬ 
um of the horizontal axis almost exclusively was characterized by the Diversity and the 
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Format/Life Situations themes, whereas the right side of the continuum of the horizontal 
axis almost exclusively was characterized by the Time and Preconceived Bias themes. As 
such, the correspondence analysis maximally separated Diversity and the Format/Life 
Situations themes from the Time and Preconceived Bias themes. Moreover, this distinc¬ 
tion means that the four themes each fell somewhere on a continuum that lay from inter¬ 
nal influences that characterized an interaction between instructors and students (i.e., Di¬ 
versity, Format/Life Situations) to external influences that characterized an interaction 
between instructors and curricula pertaining to the rapidly evolving field of mixed re¬ 
search yielding the meta-themes of internal influence and external influence. Interesting¬ 
ly, the subtheme that was clustered closest to the origin (i.e., intersecting axes) as a cen¬ 
tral subtheme among instructors was Changing Field. 

In Figure 3, participants are identified as they are positioned in the four quadrants and 
with respect to the 10 challenge themes. Specifically, one instructor is positioned in 
Quadrant 1, three instructors are located in Quadrant 2, five instructors are positioned in 
Quadrant 3, and two instructors are located in Quadrant 4. Upon closer examination of 
Figure 3, it can be seen that the one instructor who held the Multiple Paradigms concep¬ 
tual stance (i.e., MP1) is situated in Quadrant 3. In contrast, the one instructor with the 
Complementary Strengths stance (CS1) is situated in Quadrant 4. However, the two sets 
of conceptual stances that best illustrated the role that conceptual stances played in the 
formation of challenges were represented by the Dialectic stance and the Alternative Par¬ 
adigm stance. Specifically, four of the five instructors who were classified as representing 
a Dialectic stance (i.e., Dl, D3, D4, D5) were clustered together in Quadrant 3. In con¬ 
trast, three of the four instructors who were classified as representing an Alternative Par¬ 
adigm stance (i.e., API, AP2, AP3, AP4) are scattered farthest away from each other on 
the correspondence plot, appearing in three of the four quadrants (i.e., Quadrant 1, Quad¬ 
rant 2, Quadrant 4). However, three of these four Alternative Paradigm stance instructors 
were situated on the external influence side of the continuum. 

Finally, Table 4 presents keywords, overall frequencies, and frequency of dominant key¬ 
words specific to teaching mixed research and challenges among cases. Not surprisingly, 
the words qualitative and quantitative were used highly across the majority of cases. In¬ 
terestingly, instructors were invested highly in helping students overcome challenges as¬ 
sociated with a research proposal or dissertation. Also, philosophy was important to in¬ 
structors. Other words pertaining to the challenges of teaching mixed methods research 
courses were associated with methodology, such as mixing , designs , sampling, and phi¬ 
losophy. Also, instructors mentioned specific textbooks or articles associated with authors 
such as Tashakkori and Teddlie’s (2010) Sage Handbook of Mixed Methods in Social and 
Behavioral Research, Teddlie and Tashakkori’s (2009) Foundations of Mixed Methods 
Research, Creswell and Plano Clark’s (2010) Designing and Conducting Mixed Methods 
Research, and Johnson et al.’s (2007) definition of mixed research. 
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Table 4. Keywords and Frequencies Across 12 Cases Pertaining to Challenges of 
Teaching Mixed Research. 



FREQUENCY 

NO. CASES 

QUALITATIVE/QUAL 

171 

12 

QUANTITATIVE/QUAN 

130 

12 

DESIGN 

61 

10 

METHODOLOGY 

44 

10 

MIXING 

30 

8 

PROPOSAL 

27 

9 

METHOD 

24 

8 

DISSERTATION 

23 

5 

PHILOSOPHY 

23 

4 

DESIGNS 

21 

5 

QUAL 

21 

4 

PHILOSOPHICAL 

18 

7 

LANGUAGE 

15 

5 

SAMPLING 

13 

6 

TASHAKKORI 

12 

5 

STANCES 

10 

4 

INTEGRATION 

9 

6 

LEVELS 

9 

6 

TERMINOLOGY 

9 

5 

TEDDLIE 

8 

4 

HANDBOOK 

7 

6 

JOHNSON 

7 

5 

CRESWELL 

7 

4 

BACKGROUNDS 

6 

4 

PARADIGMS 

6 

4 

ONWUEGBUZIE 

5 

5 

PARADIGM 

5 

4 

DICHOTOMOUS 

5 

3 


Discussion 

The four themes that emerged pertaining to pedagogical challenges comprised 10 sub¬ 
themes, which suggests that pedagogical challenges represent a multidimensional con¬ 
struct of both internal and external influences. Moreover, the finding that these challenges 
all lie on a external-internal influence continuum is particularly noteworthy because it 
implies that instructors might consider addressing their challenges by dividing their strat¬ 
egies into short-term strategies (i.e., pertinent to the most current course) and long-term 
strategies (i.e., pertinent to future course). 
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Another important finding is that, for the most part, these challenges appear to arise as a 
function of the instructor’s philosophical stance. Indeed, this finding has intuitive appeal 
because it addresses the importance of each instructor being aware of their philosophical 
assumptions and stances prior to designing their mixed research courses. In particular, the 
finding that the instructors subscribing to the Alternative Paradigm stance were located 
far away from each other in general also has logical appeal because what members of this 
stance have in common is the belief that philosophical assumptions and stances underly¬ 
ing mixed research should be different from those assumptions and stances underlying 
quantitative (e.g., post positivism) and qualitative research (e.g., radical constructivism, 
critical theory) traditions. As such, for example, researchers who consider themselves as 
subscribing to the transformative-emancipatory stance (cf. Mertens, 2003) and those who 
consider themselves to be critical realists (cf. McEvoy & Richards, 2006) would be clas¬ 
sified under Teddlie and Tashakkori’s (2010) Alternative Paradigm stance even though 
these two stances are substantively different—likely explaining why these participants 
did not all cluster together on our correspondence analysis plot. However, the fact that 
these Alternative Paradigm instructors tended to identify challenges that represented ex¬ 
ternal influences suggests that these instructors are more likely to view long-term chal¬ 
lenges as problematic. Another interesting finding is that the participants who held beliefs 
associated with the Dialectic stance, with the exception of one participant, tended to be 
clustered together in Quadrant 3 near the origin of the plot and especially around the Di¬ 
verse Levels of Students and Applications of Knowledge subthemes, which indicates that 
these instructors identified challenges that turned out to be the two most common chal¬ 
lenges (cf. Figure 2). 

An important implication for practice is that, based on the current findings, new instruc¬ 
tors of mixed research courses potentially have numerous challenges that they should at¬ 
tempt to address when designing their mixed research courses. Interestingly, most of 
these challenges are unique to the field of mixed research. For example, although the 
fields of quantitative research and qualitative research have a long tradition, the field of 
mixed research is relatively new and thus represents a rapidly changing field (i.e., a com¬ 
ponent of the Time theme). By reflecting on their philosophical assumptions and stances 
as well as preparing examples of important keywords and concepts outlined in our study, 
new instructors might be able to predict what their most important challenges will be and 
make plans to address them to the fullest extent possible. As such, we hope that, at the 
very least, our findings at least will bring to light some of the major challenges that new 
mixed research instructors might face, as well as validate some of the challenges experi¬ 
enced by instructors who have already taught mixed research courses. 

In closing, the present study was unique in at least four ways. First, the present study ap¬ 
pears to represent only the third study—the other studies being Creswell et al. (2003) and 
Ivankova (2010)—wherein the challenges faced by instructors of mixed research courses 
have been documented. Second, this appears to be the only formal study in which the 
challenges faced by several instructors of mixed research courses were compared and 
contrasted within the same framework. Third, this study appears to represent one of only 
a few studies in which formal interviews were used to examine the experiences of mixed 
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research courses. Fourth, this is the first study, to date, to use mixed research techniques 
extensively to examine this phenomenon. We hope that through our research of challeng¬ 
es of teaching mixed research courses, the future of mixed research might be influenced 
through detailed planning, proactive strategies for addressing diversity, and careful atten¬ 
tion to the many facets important for instructors to consider toward student success. 
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